LiVLR: A Lightweight Visual-Linguistic Reasoning Framework for Video Question Answering

Video Question Answering (VideoQA), aiming to correctly answer a given question based on understanding multimodal video content, is challenging due the richness of content. From perspective understanding, complete VideoQA framework needs understand content at different semantic levels and flexibly integrate diverse distill question-related To this end, we propose Lightweight Visual-Linguistic Reasoning named $\text{LiVLR}$. Specifically, notation="LaTeX">$\text{LiVLR}$ first utilizes graph-based visual linguistic encoders obtain multi-grained representations, respectively. Subsequently, obtained representations are integrated with devised Diversity-aware module ($\text{DaVL}$). notation="LaTeX">$\text{DaVL}$ distinguishes types learnable index embedding in graph embedding. Therefore, can adjust importance when generating joint representation. The proposed lightweight shows its performance advantage three benchmarks, MRSVTT-QA, KnowIT VQA, TVQA. Extensive ablation studies demonstrate effectiveness key components